Cerebrovascular disease (CVD) and amyloid burden are the most frequent pathologies in subjects with cognitive impairment. However, the relationship between CVD, amyloid burden, and cognition are largely unknown. We aimed to evaluate whether CVD (lacunes, white matter hyperintensities, and microbleeds) and amyloid burden (Pittsburgh compound B [PiB] retention ratio) contribute to cognitive impairment independently or interactively. We recruited 136 patients with subcortical vascular cognitive impairment who underwent magnetic resonance imaging, PiB–positron emission tomography, and neuropsychological testing. The number of lacunes was associated with memory, frontal dysfunctions, and disease severity. The volume of white matter hyperintensities and the PiB retention ratio were associated only with memory dysfunction. There was no direct correlation between CVD markers and PiB retention ratio except that the number of lacunes was negatively correlated with the PiB retention ratio. In addition, there were no interactive effects of CVD and PiB retention ratio on cognition. Our findings suggest that CVD and amyloid burden contribute independently and not interactively to specific patterns of cognitive dysfunction in patients with subcortical vascular cognitive impairment.